Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

During sleep and awake rest, the hippocampus replays sequences of place cells that have been activated during prior experiences. These have been interpreted as a memory consolidation process, but recent results suggest a possible interpretation in terms of reinforcement learning. The Dyna reinforcement learning algorithms use off-line replays to improve learning. Under limited replay budget, a prioritized sweeping approach, which requires a model of the transitions to the predecessors, can be used to improve performance. We investigate whether such algorithms can explain the experimentally observed replays. We propose a neural network version of prioritized sweeping Q-learning, for which we developed a growing multiple expert algorithm, able to cope with multiple predecessors. The resulting architecture is able to improve the learning of simulated agents confronted to a navigation task. We predict that, in animals, learning the world model should occur during rest periods, and that the corresponding replays should be shuffled.Comment: Living Machines 2018 (Paris, France

Field repetition and local mapping in the hippocampus and medial entorhinal cortex

Hippocampal place cells support spatial cognition and are thought to form the neural substrate of a global 'cognitive map'. A widely held view is that parts of the hippocampus also underlie the ability to separate patterns, or to provide different neural codes for distinct environments. However, a number of studies have shown that in environments composed of multiple, repeating compartments, place cells and other spatially modulated neurons show the same activity in each local area. This repetition of firing fields may reflect pattern completion, and may make it difficult for animals to distinguish similar local environments. In this review we will (a) highlight some of the navigation difficulties encountered by humans in repetitive environments, (b) summarise literature demonstrating that place and grid cells represent local and not global space, and (c) attempt to explain the origin of these phenomena. We argue that the repetition of firing fields can be a useful tool for understanding of the relationship between grid cells in the entorhinal cortex and place cells in the hippocampus, the spatial inputs shared by these cells, and the propagation of spatially-related signals through these structures

Exposure and impact of a mass media campaign targeting sexual health amongst Scottish men who have sex with men: an outcome evaluation

Background: This paper explores the exposure and impact of a Scottish mass media campaign: Make Your Position Clear. It ran from October 2009 to July 2010, targeted gay men and other men who have sex with men (MSM), and had two key aims: to promote regular sexual health and HIV testing every 6 months, and to promote the use of appropriate condoms and water-based lubricant with each episode of anal intercourse. Methods: A cross-sectional survey (anonymous and self-report) was conducted 10 months after the campaign was launched (July 2010). Men were recruited from commercial venues. Outcome measures included use of lubricant, testing for sexually transmitted infections and HIV, and intentions to seek HIV testing within the following six months. Linear-by-linear chi-square analysis and binary logistic regressions were conducted to explore the associations between the outcome measures and campaign exposure. Results: The total sample was 822 men (62.6% response rate). Men self-identifying as HIV positive were excluded from the analysis (n = 38). Binary logistic analysis indicated that those with mid or high campaign exposure were more likely to have been tested for HIV in the previous six months when adjusted for age, area of residence and use of the “gay scene” (AOR = 1.96, 95% CI = 1.26 to 3.06, p = .003), but were not more likely to be tested for STIs (AOR = 1.37, 95% CI = 0.88 to 2.16, p = .167). When adjusted for previous HIV testing, those with mid or high campaign exposure were not more likely to indicate intention to be tested for HIV in the following six months (AOR = 1.30, 95% CI = 0.73 to 2.32, p = .367). Those with no campaign exposure were less likely than those with low exposure to have used appropriate lubricant with anal sex partners in the previous year (AOR = 0.42, 95% CI = 0.23 to 0.77, p = .005). Conclusions: The campaign had demonstrable reach. The analysis showed partial support for the role of mass media campaigns in improving sexual health outcomes. This suggests that a role for mass media campaigns remains within combination HIV prevention

A randomised feasibility study to investigate the impact of education and the addition of prompts on the sedentary behaviour of office workers

Abstract Background Office workers have been identified as being at risk of accumulating high amounts of sedentary time in prolonged events during work hours, which has been associated with increased risk of a number of long-term health conditions. There is some evidence that providing advice to stand at regular intervals during the working day, and using computer-based prompts, can reduce sedentary behaviour in office workers. However, evidence of effectiveness, feasibility and acceptability for these types of intervention is currently limited. Methods A 2-arm, parallel group, cluster-randomised feasibility trial to assess the acceptability of prompts to break up sedentary behaviour was conducted with office workers in a commercial bank (n = 21). Participants were assigned to an education only group (EG) or prompt and education group (PG). Both groups received education on reducing and breaking up sitting at work, and the PG also received hourly prompts, delivered by Microsoft Outlook over 10 weeks, reminding them to stand. Objective measurements of sedentary behaviour were made using activPAL monitors worn at three time points: baseline, in the last 2 weeks of the intervention period and 12 weeks after the intervention. Focus groups were conducted to explore the acceptability of the intervention and the motivations and barriers to changing sedentary behaviour. Results Randomly generated, customised prompts, delivered by Microsoft Outlook, with messages about breaking up sitting, proved to be a feasible and acceptable way of delivering prompts to office workers. Participants in both groups reduced their sitting, but changes were not maintained at follow-up. The education session seemed to increase outcome expectations of the benefits of changing sedentary behaviour and promote self-regulation of behaviour in some participants. However, low self-efficacy and a desire to conform to cultural norms were barriers to changing behaviour. Conclusions Prompts delivered by Microsoft Outlook were a feasible, low-cost way of prompting office workers to break up their sedentary behaviour, although further research is needed to determine whether this has an additional impact on sedentary behaviour, to education alone. The role of cultural norms, and promoting self-efficacy, should be considered in the design of future interventions. Trial registration This study was registered retrospectively as a clinical trial on ClinicalTrials.gov (ID no. NCT02609282 ) on 23 March 2015

Sustainability marketing myopia: the lack of sustainability communication persuasiveness

Sustainability communication in accommodation businesses tends to be factual and descriptive, as companies are concerned with product-based messages that focus on what they do; they appear not to understand the potential benefits of constructing messages that would influence consumers to behave more sustainably, which is effectively sustainability marketing myopia. An analysis of 1,835 sustainability messages from award-winning businesses shows that messages communicate facts not emotions, and benefits for society as a whole rather than for the individual customer. The messages are explicit, but passive and not experiential hence they positively affect the cognitive but not the affective image of the business. The lack of message normalization and customer focus reinforces the image of sustainability being a niche concern. We reflect on the reasons for these shortcomings and highlight opportunities to improve persuasive communication, which we have now applied commercially in more than 400 website analyses and 60 training courses

Theta-paced flickering between place-cell maps in the hippocampus

The ability to recall discrete memories is thought to depend on the formation of attractor states in recurrent neural networks. In such networks, representations can be reactivated reliably from subsets of the cues that were present when the memory was encoded, at the same time as interference from competing representations is minimized. Theoretical studies have pointed to the recurrent CA3 system of the hippocampus as a possible attractor network. Consistent with predictions from these studies, experiments have shown that place representations in CA3 and downstream CA1 tolerate small changes in the configuration of the environment but switch to uncorrelated representations when dissimilarities become larger. The kinetics supporting such network transitions, at the subsecond time scale, is poorly understood, however. Here we show that instantaneous transformation of the spatial context (\u2018teleportation\u2019) does not change the hippocampal representation all at once but is followed by temporary bistability in the discharge activity of CA3 ensembles. Rather than sliding through a continuum of intermediate activity states, the CA3 network undergoes a short period of competitive flickering between pre-formed representations for past and present environment, before settling on the latter. Network flickers are extremely fast, often with complete replacement of the active ensemble from one theta cycle to the next. Within individual cycles, segregation is stronger towards the end, when firing starts to decline, pointing to the theta cycle as a temporal unit for expression of attractor states in the hippocampus. Repetition of pattern-completion processes across successive theta cycles may facilitate error correction and enhance discriminative power in the presence of weak and ambiguous input cues

Visual arguments in film

Nuestro objetivo es señalar algunas diferencias entre los argumentos verbales y visuales, y promover la perspectiva retórica de la argumentación, yendo más allá de la relevancia de la lógica y de la pragmática. En nuestra opinión, si ha de ser racional y aceptable como argumentación (visual), un film debe dirigirse a espectadores que tienen creencias informadas sobre el tema visto en la pantalla y sobre las limitaciones y las convenciones del medio. En nuestras reflexiones, aplicamos el análisis retórico al cine como un acto simbólico, humano y comunicativo que a veces puede entenderse como un argumento trazado visualmente. Como mezcla de estímulos visuales, auditivos y verbales, el film exige una interpretación y una (re)construcción activas y complejas. Nuestra sugerencia es concentrarse en cinco elementos diferentes, pero relacionados entre sí. La reconstrucción y la evaluación del argumento visual se basarán en esos elementos, y todo el proceso constituirá una argumentación visual.Our aim is to point out some differences between verbal and visual arguments, promoting the rhetorical perspective of argumentation beyond the relevance of logic and pragmatics. In our view, if it is to be rational and successful, film as (visual) argumentation must be addressed to spectators who hold informed beliefs about the theme watched on the screen and the medium’s constraints and conventions. In our reflections to follow, we apply rhetorical analysis to film as a symbolic, human, and communicative act that may sometimes be understood as a visually laid out argument. As a mixture of visual, auditory, and verbal stimuli, film demands active and complex interpretation and (re)construction. Our suggestion is to focus on five different but interrelated elements. The reconstruction and evaluation of the visual argument will be based on those elements, and the whole process will be one of visual argumentation

When Do Objects Become Landmarks? A VR Study of the Effect of Task Relevance on Spatial Memory

We investigated how objects come to serve as landmarks in spatial memory, and more specifically how they form part of an allocentric cognitive map. Participants performing a virtual driving task incidentally learned the layout of a virtual town and locations of objects in that town. They were subsequently tested on their spatial and recognition memory for the objects. To assess whether the objects were encoded allocentrically we examined pointing consistency across tested viewpoints. In three experiments, we found that spatial memory for objects at navigationally relevant locations was more consistent across tested viewpoints, particularly when participants had more limited experience of the environment. When participants’ attention was focused on the appearance of objects, the navigational relevance effect was eliminated, whereas when their attention was focused on objects’ locations, this effect was enhanced, supporting the hypothesis that when objects are processed in the service of navigation, rather than merely being viewed as objects, they engage qualitatively distinct attentional systems and are incorporated into an allocentric spatial representation. The results are consistent with evidence from the neuroimaging literature that when objects are relevant to navigation, they not only engage the ventral “object processing stream”, but also the dorsal stream and medial temporal lobe memory system classically associated with allocentric spatial memory